Search for: All records

The instruction hierarchy, which establishes a priority order from system messages to user messages, conversation history, and tool outputs, is essential for ensuring consistent and safe behavior in language models (LMs). Despite its importance, this topic receives limited attention, and there is a lack of comprehensive benchmarks for evaluating models’ ability to follow the instruction hierarchy. We bridge this gap by introducing IHEval, a novel benchmark comprising 3,538 examples across nine tasks, covering cases where instructions in different priorities either align or conflict. Our evaluation of popular LMs highlights their struggle to recognize instruction priorities. All evaluated models experience a sharp performance decline when facing conflicting instructions, compared to their original instruction-following performance. Moreover, the most competitive open-source model only achieves 48% accuracy in resolving such conflicts. Our results underscore the need for targeted optimization in the future development of LMs. 

Measurements are presented of the cross-section for the central exclusive production ofJ/\psi\to\mu^+\mu^- $J/\psi \to {\mu }^{+}{\mu }^{-}$and\psi(2S)\to\mu^+\mu^- $\psi \left(2S\right)\to {\mu }^{+}{\mu }^{-}$processes in proton-proton collisions at\sqrt{s} = 13 \ \mathrm{TeV} $\sqrt{s}=13TeV$with 2016–2018 data. They are performed by requiring both muons to be in the LHCb acceptance (with pseudorapidity2<\eta_{\mu^±} < 4.5 $2<{\eta }_{{\mu }^{\pm }}<4.5$) and mesons in the rapidity range2.0 < y < 4.5 $2.0<y<4.5$. The integrated cross-section results are\sigma_{J/\psi\to\mu^+\mu^-}(2.0 ${\sigma }_{J/\psi \to {\mu }^{+}{\mu }^{-}}(2.0<y_{J/\psi }<4.5,2.0<{\eta }_{{\mu }^{\pm }}<4.5)=400\pm 2\pm 5\pm 12pb,{\sigma }_{\psi \left(2S\right)\to {\mu }^{+}{\mu }^{-}}(2.0<y_{\psi \left(2S\right)}<4.5,2.0<{\eta }_{{\mu }^{\pm }}<4.5)=9.40\pm 0.15\pm 0.13\pm 0.27pb,$where the uncertainties are statistical, systematic and due to the luminosity determination. In addition, a measurement of the ratio of\psi(2S) $\psi \left(2S\right)$andJ/\psi $J/\psi$cross-sections, at an average photon-proton centre-of-mass energy of1\ \mathrm{TeV} $1TeV$, is performed, giving$ = 0.1763 ± 0.0029 ± 0.0008 ± 0.0039,$$ where the first uncertainty is statistical, the second systematic and the third due to the knowledge of the involved branching fractions. For the first time, the dependence of theJ/\psi$ $J/\psi$and\psi(2S) $\psi \left(2S\right)$cross-sections on the total transverse momentum transfer is determined inpp $pp$collisions and is found consistent with the behaviour observed in electron-proton collisions.